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ABSTRACT 

Because the attitude .research approach to determining 
publiq opinion typically yields data irrelevant to public policy 
formation, researchers need to consider nontraditioi^al approaches to 
public opinion measures. Attitude research assumes that people 
actually have readily measurable opinions and that each attitude can 
be measured singly. Measurement is directed toward uncovering a 
perison*s most preferred 'position. To be politically relevant, ^ 
however, survey data must indicate the intelligibility of public 
responses; require respondents ^to set priorities and make .trade-off s; 
depict what costs would be tolerated for a given policy; and 
distinguish degrees of political acceptability. Alternative measures 
include the "budget pie" technique in which participants are asked tp 
allocate a fixed amount among different pjrograms. A second technique 
involves providing participants with chips that they allocate to 
slots which represent government programi^. Disadvantages of these ^ 
methods include the amount of time required and the capacity of 
citizens to make resource allocations. A third approach allows 
respondents to select ranges ^H|^sp9nses categorized according to 
acceptance, indifference, and^^jection. Aggregation of data in all 
of these approaches is a major problem. Nonetheless, the goal should . 
not be to measure public attitudes, but rather, political pre&erenca. 
(Author/KC) 
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A b s t r a*c t , 



Assessing the relationship between public opinion and public 
policy is a central question in public opinion research. Unfortunately, 
because most of the opinion data we collect is rooted in an "attitude 
research" approach, these data are^^equently irrelevant Jto assessing 
an opinion-policy relationship. To be politically relevant,/ survt^y 
data must: (1) indicate^ the intelligllbility of public responses not 
just the reliability and validity -of the responses; (2) require 
respondents to set priorities and make trade-offs across policies; (3) 
depict what costs would be tolerated for a given policy and (A) 
distinguish degrees of policy acceptabilify^ , In addition, the data 
must be aggregated else the message convened to leaders will be highly 
idiosyncratic. Survey techniques not rooted in attitude research 
(e.g., budget pies) are only partially successful in providing 
• politically relevant policy preference; data. In light of these 
problems either we must expend considerable effort re-tooling our 

« 4 

instruments or Lhe focus of our analysis must be changed. 
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XSv^the government enacting po!(.icies desired by the people? 
This i/ an important .and frequently asked question. For ^hose 

viewing democracy as popular control of public policy, this 

/ 

opinion-policy relationship is the fundamental question. It is 
also relevant for scholars concerned with governmenL 
accountability. Likewise, analysts investigating 
representational relationships such as those between citizen and 
legislator must also assess the degree of opinion-policy 
agreement. Even practicing politicians must come to grips wiDi 
this consistency issue in deciding what policy stands to take to 
eaiiance their chances of reelection. Though the opinion-policy 
relationship has drlwn the explicit attention of only a limllcd 
number of scholars Ve.g. , Devine, 1970; Monroe, 1979; Weissberg, 
1976), it is clearly one of the most basic issues in public 
opinion research. 

Scholars concerned with this opinion-policy relationship havfe 
had to resolve two major problems. The first; and most basic, is 
the theoretical connection between mass opinion and governme'nt 

policy. Typical here are discussions involving the 

\ 

differentiation of the public (elites, "issue publics" et£. ), the 
possible flows of influence between leaders and citizens, and the 
role of intervening organizations and \nstiturt:ions in this 
linkage, '^e second problem^ that has to. be 'addressed is the ^ 
measurement of public opinion. Of the two problems, this appears 
to be the easier to solve* Ascertaining what the public wants is 
treated largely as a techaic-al problem— the perceived obstacles 
concer^l such things as sampling, question wording, scale 



) 



construction, and the proper statistical techniques to be 

* 

employed* It is assumed that we know what is to be measured so 

energy should be therefore directed to largely technical questions 

of how opinion is to be ascertained. 

This paper will argue that the currently *populaQnethod of 

measuring public opinion is largely inappropriate for collecting 
data on th^ relationship between popular preferences and public 
policy • Existing methods of collecting opinion data are generally 
incapable of telling us what the public really wants regardless of 
response rate, size of sampling error, or cace exercized in 
question wording. Put bluntly, our contention is that we are 
looking fot the yrong kind of information whea we ask respondeiiL 
questions like "Do you think ^the government should spend more (or 
less) money on military defense?" At best, current polling 
techniques yield crude approximations of the public policy desired 
by citizens. What is needed is a fundamental re-jff ocusing of our 
opinion measurement efforts, not a technical perfection of the ^ 
existing approach. 

Our discussion of ascertaining what the public wants from 
government will be divided into two parts. First, we shall critically 
exam'ine the existing approach to ascertaining public opinion. Among 
other things, we shall show that displaying public thinking ^j^^ issue 
is not identical to showing what>. the public wants from government on 

" that issue. Second, some alternative approaches to ascertaining 
public issue preferences will be considered. This will entail 
suggestions on bot%what is to be measured and how this might be d^ne 
using convent lonaij^aiaple surveys, 

^ 
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The Conventional Approach to Measuring Public Opinion 

When most researchers set out to ascertain what citizens want from 
their government they use an approach that has its theoretical and 
methodological roots in what might be called "attitude research." (3ee 
Chisman, 1976, Chs 2-3 for a further analysis of "attitude research.") 
This "attitude research" approach is not the same thing as "survey 
re'search" though to political scientists the two are almost 
inseparable. "Attitude research" represents an integrated view of vihat 
people think, how this thinking can be measured and how millions of 
separate thoughts can be combined into colle9tive portraits of "p^lic 
• thinking." "Attitude research" is a viewpoint, not a colle^ll^ 
measures and techniques. Like the official reiigion in a successfull 
theocracy, this attitude research approach to public opinion is so 
deeply ingrained that most adherents are oblivious to even the idea 
of an alternative. 

This attitude paradigm contains several elements that are 
imjJortant for our analysis: 

1. Citizens possess — or can readily possess — numerous genuine 
attitudes on a wide variety of political topics, issues; or proposals. 
The capacity of citizens to hold a large number of "real" atti tildes 
follows directly from the operational definitions of attitudes and 
their j>roper measurement. That is,* citizen thinking cannot be directly 
observed or/measured. But, responses generated by an external stimulus 
(i.e., a question) can be recorded. The character of the underlying 
mental state generating th^ response has nothing to do with the validity 
or reliability of .the response. All that raatterb for the creation of 
valid and reliable attitude data is that the overt response ^ 



systematically corresponds to some (unmeasured, undef ined)%iriderlying 
mental phenomenon. Given ample opportunity to improve one's questioas 
and a willingness of people to react, a researcher can readily show 
that'citizens possess valid, reliable attitudes on almost every subject, . 
imaginable. Attitudes are "real" because- they yield me.nsuremeats that 
are valid and reliable. It is inconceivable from the perspective of 
the attitude measurement paradigm that citizens have nothing of any 
importance to say. As far as each citizen is concerned: "I think, 
therefore, I have political attitudes." 

2. , Attitudes are properly measured one at a time. To determine 
what a person thought, a researcher would ask numerous questions each 
dealing with a single issue topic . Of course, the responses may be 
grouped into elaborate structures and configurations, but such 
inter-relationships occur after the single attitude has been measured. 
Typically, a respondent is asked questions like "Do you favor increased 
defense spending?" or "Should more aid be given to the poor?" Only 
rarely are attitudes about two or more topics examined simultaneously. 
("Contamination" is a serious sin in the attitude measurement 
approach.) < 

m 

3. Attitudes are viewed as being arrayed along some dimension and 
the* purpose of a measurement is to determine the particular point on 
the continuum that best reflects a person's thinking. The dimension 
can be in terms of agreement level with a statement-Defense spending 
should, be increased— Strongly agree, Agree, No Opinion, et.c. It 
could be in terms of one alternative' in an array-Defense spending 
should be cut. uot changed, increased?. Perhaps even a seven point 
scale. In any case, however, mbasurement is directed to finding the 
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one position that best reflects what a person feels or wants given a 
series of choices on some dimension. Discrepi^ncies between a person's 
precise, position an(J what the measuring instrument yields exist, of 
course, but these discrep-^ncies are viewed either as measurement error 
or a price to be" paid for simplifying highly complex issues. 

4. The primary purpose of attitude measurement research is to 
describe in a comprehensive fashion public opinion kt the individu-al 
level. This involves both the accurate sampling of the general 
population and the asking of as many questions. as possible. Good 
polling can be likened to taking a series of de,tailed snapshots of a 
group withwnobody left out. Answers to, say. 100 questions gives a 
better picture than, say, responses to 10 questions. 

From the perspective of an opinion pollster these four 
assumptions-people do have real, readily measurable opinions, each 
attitude is to be measured singly, measurement is directed to 
uncovering a person's most preferred positions and the more 
questions the better thj picture of public opinion— are self-evident. 
If. however, we were Co put ourselves in the position of a public 
official looking to the public for guidance, a §tudy incorporating 
these assumptions would usually yield troublesome or even i^refevant 
results. It is not that these assumptions lead to meaningless data; 
rather, the resulpant information is often inappropriate or 
incomplete.' 

Consider the question of wh^t constitutes a valid, reliable 
attitude. It is quite likely that much valid, reliable opinion poll 
data are invalid with respect to government action. For example, a 
citizen may desire k policy that is impossible to implement (e.g., 
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eliminate all pollution). Or, the preference may be hopelessly vague 
or ambiguous (e.g., leaders should represent the people not the 
•'special interests'*). Even a seemingly valid preference may be 
irrelevant if it is based on incorrect premises (e.g., reduce taxes for 
poor people by increasing the size of the standard deduction). A 
valid, reliable citizen preference (from the perspective of the 
attitude measurement paradigm) can also be politically hopeless even if 
it was doable in principle and based on correct ^ctual premises (e.g. > 
reduce the threat of nuclear war by iiisti tutting UN supervised world 
disarraaraeut). In short, valid, reliable citizen attitudes need not 
make an^^olitical sense. 

The practice of treating attitudes as separate entities to be 
mea^i<^ one at a time is also at adds with the types of c)^ice& 
confrontedN^yl public officials. To public officials, the ptl^it of 
one policy is inevitably linked to the petrsuit of other policies. 
While two issue domains may be conceptually distinct, they may be 
ihtiraately intertwined in terms^of their accomplishment. The conflict 
between lower consumer prices and protecting U.S. industry from cheap 
imports is an obvious illustration. Moreover, policies that are not 
inherently contradictory may in fact be mutually exclusive given 
political bargaining and coalition building. For example, a very' 
liberal legislator might be able to marshall a majopiLy for one or two 
liberal proposals, but it is very unlikely that vote trading could be 
successful for several very liberal proposals. Thus, while the usual 
public opinion polls ^llow citizens that luxury of not having to worry 
about contradictions, trade-offs, and bargaining, public officials do 
have" these worries in their decisions. 
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Regarding the third assumption of the attitude measurement 
paradigm-— measurement is supposed to, locate an individual's single most 
prefered position on some dimension — this too is i,^elevant to the 
types of choices faced by public officials. Rarely, for all sorts of ^ 
fairly obvious reasons, do individual officials and governments in 
general expect to gain their most pref e^d alternative a significant 
portion of the tilne. An official viewing his or her options is likely 
to think in terms of best possible outcome, minimally satisfactory 
outcome or even least objectionable outcome • While a respondent in n 
typical mass survey essentially makes black or white choices, public 
officials usually operate in a world of numerous shades of gray. 
Moreover, while the goal in the attitude measurement approach is to 
create uni-diraenional scales, issues faced by public officials are 
frequently multi,-dimensional. Indeed, by habit and inclination most 
professional politicians readily see more than one issue in what 
publicly is merely a single issue. 

Finally, while reponses on a wide range of issu.es f rom ^a 
representative 'sample may be the final objective from the attitude 
measurement perspective, such information hardly begins to inform a 
decision-maker concerned with public opinion. Given the time 
consuming na^re of changing policies, the likely long term 
persistence of opinions is xin important (and rarely ascertained) pioce 
of information.. Obviously, just how one reacts to a public dt-mand for 
increased defence spending can greatly depend on the perceived 
fickleness of this^ demand (and standard measures of question 
reliability hardly tap such long term endurance). In addition, 
typical survey items rarely provide the level of detail on just what 
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the public wanes or how it is to be accbraplished. For example, a 
typical survey question showing public support for greater government 
assistance for the poor can be interpreted to mean a call for almost 
any type of action — eliminating all assi^^tance programs so as to 
encourage greater self-reliance to guaranteed incomes of $50»000 per 
year all "help the poor-" Clearly, then, a public official given the 
results of a typical poll covering even 100 public issues is provided 
little (if any) intelligent guidance in givirlg people what they want 
on any one issue • 

Wh a t_sh o u 1 d opinion pol ls measure ? 

Our argument thus /ar is that C'hc common public opinion poll Is 
rooted in an attitude measurement approach and this approach ruadil)f 
leads to the collection of data tHat are largely irrelevant to the 
political prpcess. As a result, in most situations survey data do not 
(1) provide realistic messages leaders regarding, what the public 
wants and (2) permit a judgment on whether public preferences and 
government action are^in agreement.' What we shall suggest here are 
some of the things that should be ascertained by surveys if such data 
> / are to be politically relevant. Some of the problems in obtaining 

relevant data will also be considered. * As we shall see, determining 
what the public "really wants from government'' is far more difficult * 
and complex than assessing public attitudes towards hundred of 
political objectives, 
t 

Ascertaining In tell ig able Opinion 

Perhaps Che first objective in a poll designed to determine what 
the public wants is to have some way, of deciding When the public has 
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nothing relevant or ' intelligent to say. Recall that the'attitude 
measurement approach can almost always generate reliable and v<ilid 
responses on nearl^r'any political topic imaginable (for an analysis of 
what people are willing to offer, interviewers, see Bishop^et^;^, 
1980 and Schuraan and Presser' 1981, th.-5). Yet. the idea of the mass 
public being, able to participate "intelligently in the debate aver any 
government pol^kv is clearly nonsense. At present, detecting that tlie 
[Public "speaks" ^bberish on an Issue is difficult. Contradictory 
response patterns, low rates'of passing informational or interest 
filter questions,^ and high "Don't know'' respon'se rates are only little 
morff thjin rough approximations. , . 

Suppose analysis did start with the question "Docs the public have 
anything intelligent to say?" rather that "What do'es. the pu^'blic say?" 
how does one proceed? If we begin with th'e assumption that the purpose 
_of the poU is to measure what currently exists, an elaborate series of 
filter questio-ns would seem the appropriate mechanisms for separating 
sensible from nonsensical opinion. Such questions would sort people 
out on the basis of such charactefistics as in|orraation level, capacity' 
to analyze policy consequences, understanding the relevant livpits on 
decision-makers and the like, , if one began with the^ assumption 
that most citizens could offer sensible opinions if given a chance, 
-each survey question would be accompanied by a crash educational course' 
on the topic. This might include a review of pertinent facts. 
argument^of;fered by experts, and 'estimations of possible -consequences 

of a given policy. 

Though both strategies woulc/ v^ld "better" mass opinion, such 
improvements have their. costs. ExcVfo"^ the simplist issues, the use 
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.of a rigorous sec of ■ hurdles" to separate sensible f torn- unsensible;- " . ' 
opinion would probably eliminate most resRondei^ts. "PubUc opir^on.". , 
using 'this approach might- consist of Dhe opi'nion of, a qu^vrter'-or- Uess.- - 
"of the eotai adult population. ..Providing a cjrash course oa ^n. ts^sue . 
would surely increase the number, of people .with intelligible opinions, 
but thi« etlucational. service would place heavy burdens -on b<ith-\ 
interviewers and polling organizations. >he precise cqntieric of .such^ 
instant education could also be a troublesom4 issue since' reasonable - 
people can differ over such things W. the possible 'im^p^'ti of e^i?olicy, 
its Wal cost or its feasibility. Being balanced^ and ob*jectiv6 would 

» **• 

not be easy. ' , ' , • j " . ,■ ' 

Requi r_e_responde_nts3o^ make- J.ra^^^^ ' ^ ' 

second objective in creating, a politically relevant measure of public • 
bpii^ion is'to forge respondents to^cide on tra;de-off6 and order 
priorities. As we suggested in our discussion of the ff«itude. . 
measurement approach', respondents in surveys, unlike public official^, 
can usually make choices as if everything could be done at once. 
Obviously, if mass preferences and gover^^merit decisions are to be 
compared, similar sets of constraints must apply to both. Respondents 
must, for example, decide whether government efforts to reduce 
unemployment will come before efforts to reduce inflation and at 
what levels will these priorities be changed. 

In some ways the introduction of priority and trade-off decisions 
with surveys is- quite simple. Questions like "Which is the most 
pressing national problem?" or "What should come first-safe cars or 
cheap cars?" are ^asy to ask. Yet. as was true in ascertaining a more 
intellagable public opinion, several problems remain to be resolved. 
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One such problem concerns the capacity of citizens to^^elect* prior ities 
or raak^ trade-offs invol\^ing important policies given virtually no 

attention in current political debates. For example, the federal 

y 

government's financial and regulatory involvement in education is a 
significant policy that is almost unconsciously accepted by citizens. 
Few people have given serious attention to what might happen it this 
involvement were sharply curtailed. Indeed, oVen expei^ts would be 
hard pressed to predict the consequences^^ a ma jor* policty change. How 
then do we ask citizens about giving up. the benefits of federal money 
in excharige for greater local control? Can people even begin to 
comprehend *the mean-ing of such' a shift? Similar difficulties can 
easily emerge when ask4ng citizens about altering the^ method of 

selecting public officials, major changes in methods of taxation or 

' . ^ > J ^ 

revamping the principles upon which U^S. foreign policy is based. 

Low salience but very important policies are especially difficult 

to handle when respondents are givens relatively unstructured 

opportunities to set government priorities. It is har'd to imagine 

people telling government to stick with its on-going, non-publicized 

well-established functions rather than re-allocate resources into mor^e 

topical,, better publidized areas. For example, in the 1970s few 

citizens would have placed, say, maintaining a stable financial system 

ahead of reducing pollution on their agenda of policy priorities . Yet» 

it seems protoble that most citizens would "really" prefer a stable 

financial'' s/stem over cleaner air if they were somehow forced Jto, 

iinagine the consequences of, say, numerous bank failures, a return to 

unregulated bank note currency and so on» Letting citizens set 

priprities via surveys can easily produce a government soley concerned 
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with a steady st,reara of new well-publicized but relatively so^all 
problems. Major policies now taken for granted would be to be 
ignored. 

Establish cos» ts to be tolerated. Public officials know full 
well that there is no free lunch in enacting program. New policies 
must be paid Tor not only in money, but in negative outcomes in other 
areas, as well. New four lane expressway means higher taxes, the 
confiscation of private property, a greater likelihood of noise and air 
pollution, possible dajnage to wildlife areas, and perhaps the weakening 
of established neighborhoods. Such costs, while apparent to officials 
wha must make decisions, are rarely presented to respondents on 
surveys. Respondents can typically select4j* policies as if c^iey were 
free, a procedure that can easily create the impression that the 
public is clam^oring for all sorts of new programs. 

As was true in asking ci^tizens to make trade-offs and order their 
priorities, a solution to this problem sterns relatively simple and 
straight-forward. One could imagine questions such as ^'Should the 
gc^vernment spend $2 .billion to impfove our highways?"" or "Would you be 
willing t^ have your taxes increased 20 dollars a year to get better 
highways?" Or open-ended, less structured questions could allow ^ 
respondents to put their own' acceptable price tags on policies. ^ These 
types of questions > however, only appear to yi^ld realistic inforiuation 
of what citizens are willing to pay for certain polici^w;^ 

One obvious problem concerns asking people to deal with huge 
numbers well-beyond their comprehension. Even experienced raembefs of 
Coijgress acknowledge the dl^fficulty of understanding a billion dollars. 
let\ 'alone budgetary items that run into the tens of billions. 
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Simplif ication^^fforts typidally^ alao yield choices well b^ond the 
grabp of raosc people (for example, should the Navy build iwo new siiper 
aircraft carriers or should we have ten new fully equipped hospitals)-. 
Perhaps the only way around this pcoblera is to ^ran^slate all monetary 
costs into sums that make sense do people — $1.98, $20, e^c. 

.... j . - 

A second, 'and somewhat, rel^atdd, problem concerns getting survey 

< > 

respondents to distinguish between an acceptable total cost Cor a 

given policy versus <in acceptable m a rgiha 1 cost. Consider the 

{ 

following* two questions: '*Would you be willing to ^ave $10 of your tax 
mdtiey go to f^ed hungry ' school chj^ldren?" and "Wdffld .you be willirtg to 
pay $10 more.An taxes in order to feed hungry J&hool children?" The 
^ first question assumes a fixed amount of tax payment cvnd calls for 
'/allocation witlun that amount. Endorsemeats of worthy endeavors such 
as helping the needy* are easy to make given that the tax money must be 
given to the government anyway. In the second case, however, the 
respondent is being ksked to add an additional $10 to his or her tax \* 
J)Mrden.^ Forced with an even larger tax bill, enthusiasm for helping 
the needyvmay quickly cool. 

Problems become more severe when we move away fcoin purely monetary 
costs. While policy-makers are likely to recognize the hon-mon<fctary 
costs of a given policy^ such costs are commonly unknown to typical 
citizens). For example, how many people are likely to ^think through the 
full cos^ts of using more U.S. mined coal rather than imported oil? 
Moreover, even if such costs are spelled out by the interviewer, they 
may be difficult to comprehend (e.g.. the costs of respitory ailments 
created by burning coal near cities). And, needless to say, reasonable 
people — intrerviewer and interviewee alike — can disagree over the costs 
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of policies.' As a rfesulc, support for proposed policies may rest on, 
quite different estimates of what these actions are goirig to cose. How 
does a public officiral respond to a call for action if such a call, in 
the official's estimation, is based .upon unrealistically low cost 
estimates? What if this call is based on the mistaken belief on the 
part of each citizen that they personally can escape the costs of a 
policy while reaping the benefits? 

Dj^n^ulsh.Amon D egrees of Policy Acceptability . The yltiipate 
goal of the existing "attitudte" approach to publJLc opinion measurement 
is to ascertain /6he respondent most preferred positions on an issue. 
To be sure, problems o'f question design'and time limits on intervievAs 
prevent complete success, but the goal remains that o£t deueriuiniag> what 
a person wants most. As previously noted, one's "most preferred 
''position" is only part of a discussion of political alternatives among 
public officials. Equally relevant to officials are preferences thJt ♦ 
can be given names such .as "reasonably acceptable," "barely acceptable" 
"equally good and bad" "unacceptable" or "catastrophic" Since public 
officials frequently must work for less than optimal outcomes, it ,is 
only fair that ordinary citizens consider choices in terms of ranges of 

Ifc acceptability. t 

Several possible solutions exist. Respondents can simply be asked 
to sort policy choices into categories such as "most preferred'-' or 
"unacceptable under any condition." Or, the technique of paired ^ 
comparison can be •enfi'loyed to uncover a hierarchy, of preferences (all 
possible pairs of alternatives would be judged two at a time in terms 
of "accept" or "reject"). A, related methodology is the Q sort (boCh 
paired comparison and the Q sort are described in Kerlinger 1973, 
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506-7, Ch. 3'4) A more complex solution is co ask respondents 
how tl\ey might react if a particiular policy was put on the 
public agenda. Responses could range from "riot" or "immigrate" tc 
"send financial support to advocate." Degrees of acceptability might 
even be put into purely monetary terms— how much of your disposable 
income would you be w^^fling to spend to enact (or block) a particular 

policy? , » 

Though in principle distinguishing among degrees of policy 
acceptability is not difficult, in practice several problems must be 
resolved. One such problem concerns whether people can make real 
distinctions among numerous non-optimal alternatives* Especially 
where an issue has .not been carefully considered, diverse alternatives 
can easily be lum^ped together so attempts by th^ respondent to separate 
one sub-optimal. pre^ference from another will be unsuccessful ("They're 
all bad!"). In addition, a person's categor iation of options into, 
say "preferred," "acceptable" or opposed, may vary according, to ^ 
judgments on what is politically possible. Consider the prefer^c^s of 
a 'very anti-black public official and a member of the Ku Klux Klan, 
The former, understanding what is possible, may view the option "make 
affirmative action purely voluntary" as an "acceptable" race relations 
policy. The Klansraan, however, lacking an understanding of what could 
be done sees anything less than enforced re-segregation as 
unacceptable- Both may in fact share tihe identical preferences* 

Finally, as 'several psychologists/ have noted (^see, for example, 
Sherif and HoVland, 1961), judgments regarding acceptability of a 
particular position can greatly be affected by character of^the options 
presented (the so called "assimilation-contrast ef^fect"). 
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Consider, for instance, a range of alternatives from ^^to 10 with 3 
being I person's most preferred choice. Given a choice among, say, 1. 
2, 3, A, 5, 1 and 5 may be labeled as "unacceptable." But if the 
options 9 and 10 were added, options 1 and 3 may "move" towards, 
acceptability. In other words, an unacceptable option can suddenly 
look a lot better when considered in the context of a truly awful 
alternative. Generating valid responses i$ further complicated by the 
likely absence of agreement among researchers regarding which 
particular policy alternatives should be included-^-'including several 
extreme options could easily make almost any presently discussed policy " 
"acceptable" to most respondents. 

Create__a„cons_ensus..on wh_a^t^^ Thus far we have 

considered what types of information had to be gathered from 
individuals. *,a' useful and politically relevant techniqxie must also* 
aggregate this Individual level data into some overall consensus. 
Unless this were done, public opinion- vould be so^idiosynciat ic that 
.satisfying it would be Impossible. Consider the problems faced by a ^ 
public official who is told that while there i.s mu<* inte/ligi ble. ^ 
do-abyLe opinion,' enormous differences exist over priorities. • - 

"\rade-offs, 'what i^eo pie ar,e willing to pay and what options are deemed 
. to be acceptable outcomes. In this situation, no c&i^ of action 
^couM please.-a nfajority'. Reaching a judgement regarding policy-opinion 
'consHsVehcy would b. nearly impossible. The public must speak and 
speak we\l, but the messages must also add up to a clear preference. 

Among decision-makers the problem of creating a consensus is 
usually solved by "politiking"-pressures. arguments, side-payments, 
threats of force, manipulation, and the like., Recent congressional 
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actions on President Reagan's proposal budget well illustrates how 
enormous differences ofV^pinion can be aggregated. Individual 



citizens offering their opinions independently of each other, do not 
face these pressures, hcwever. Their responsibility is to say what 
They want from governraeJt , not provide what they believe to be a 
consensus acceptable to /themselves and numerous other citizens wha^ they 

have never met. , 

Obviously, it is the responsibility of the researcher tx> create a 
consensus out of all the diversity. Using the common attitude 
measurement approach this consensus is acheived quite easily. 
ReVondents can be offered dichotomous choices or responses from several 

A 

questions are combined into scales and scale position can be collapsed 
to yield an apparent consensus. Indeed, recent years has seen an 
explosion'of sophisticated stati|^tical techniques desigty to reduce S_ 
large data structures to raor.e manageable proportions (e.g.. factor 
analysis, smail"^ space analysis, etc.). » ^ 

Once we move away from this sitapie c^ollectidn of attitude data, , 

' C ' ^ >••• • . 

howevej, the means\o create' this consensus are less clear. How might 
you ^combine" the responses of the following two individual^: 
Individual A favors spending an additional' $ 10 billion on national 
defense, sees this as the nation's first priority, is willing to accept 
some specified level of inflation and deficit spending as a 
consequence and would accept a $5 billion increase as minimally 
acceptable. Individual B likewise favors $10 billion more for defense 
but only (1) spending for social services is also increased by $10 
billion, (2) these increases are funded by increased taxes on large 
incomes, and (3) the money is spent disproportionately in areas of 
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high black unemployment . Moreover, individuals B is willing to accept 
a zero increase as politically acceptable. Compareri to. summing 
• responses across scale items, aggregation here is like adding apples 
and oranges. 
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Ef f or_ts_ to_achervejolj^t^allir_ relevant p referen ce measu^res ^ . 

Almost all the polling presently conducted is based on what we ^ 
have called the "attitude research" approach. That is. respondenLs are 
given statements or questions about a particular fi§sue and they pick 
one of several possible alternative options. This response, so long 
as it is judged reliable and valid in terms of an unobserved ' 
orientation, is considered a datum indicating what people want from 
government. Our contention has been that this approach typically 
yields data irrelevant to assessing a public opinion— public policy . . 

relationship. Wq, shall now briefly review some noSSftgadixional 
approaches to survey research that may offer help. in creating 
• /politically relevant measures of public opinion. 
) One survey technique that holds some promise for generating 
politically relevant data is the so called "budget pie" technique. 
Several researchers have experimented with this technique and in some 
instances the techniques offers respondents a much more politically \ 
realistic set of circumstances than the typical poll question. 
Basically, a respondent is given a finite set of resources (graphically 
portrayed as a pie) and asked to%llocate this fixed pie among 
differ^t programs. The respondent is thi^ forced to limit demands and 
(implicitly) make trade-offs. For examjJe, Mclver and Ostrom in 
their 1972 survey of the St. Louis metropolitan area present a "pie" 
marked 0 through lOOZ and ask respondents to allocate 
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police expenditures or their jurisdiction among patrolling^ detective work and ^ 
admirtistration. Similarly, May (1982) using telephone interviews of 
Walnut Creek^ California residents has them allocate affixed budget 
across five city supplied services (e.g. , police,' streets* recreation, 
e t c • ) • 

A more complex version of this budget pie techniquetis offered by 
Beardsley, J|^ov;/nack! and Reynolds C197A), In one version respondents 
are given a game bcJard with Vb slots, each representing a governemnt ^ 
program (e.g,, national defense, public ec?ucation). Along with a brief 
description of each slot, ^xi-sting government, expenditure levels are ^ 
Indicated. Respondents are given 100 chips, each representing a penny 
of an average tax dollar and are asked to allocate these' chips amoni; f 
Che expenditure categories. In a. second version respondents are given 
Che option of adding or substracting 50 chips thus ^increasing (or 
decreasing) both overall expenditures ,and tax bills, beardsley o i. 
al. also describe a study by Hoinville (1971) that deals with 
priorties and trade-offs in selecting 'housing. Here respondents could 
allocate finite resources among amenities such as a quit^ neighborhood, 
safety or travel time to work. To fnsure that resources would not be 
allocated solely to the most attxfactive amenities, minimal resource 
allocations were also required. 

Though this "cut a budget pie" has some ^^^^ advantages over the 
more traditional *'Do you want more (or less) money spent on X?" it is 
not without some problems. First, it is questionable whether the 
technique can han^Te^^^large number of fairly specific issue calces. 
When dealing with something comparatively simple, e.g., factors in 
buying a house as in the Hpinville study described abo>^e, allocating 
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-resources akong 15 or so areas can pretty well exhaust the universe of 
policy choices. Jipwever, when setting priorities and making 
trade-offs on national issues, th,ere ate hundreds" if not thousands of 
policy areas to consider. Perhaps the only realistic alternative is 
to have respondeij^^make choices within a general issue 'domairv? 
Instead of the general categories used by researcher like Beardsley 
et. 4I. — national defense vs. health vs. housing and community 
development — choices n1a7.be made within a category (i.e.-, more missiles 
vs» a larger .standing army). Needl^g^ to say^ while more realistic and 
yielding more precisfe information, this strategy side-steps crucial 
choices across different policy domains (e.g. guns v^. butter). 

A related problem concerns the capacity of m<iny citi/Vens, 
especially those of limited education, to make tw^"^ elementary 
resource 'allocations. Mclver and Ostrom report, for example, that 
only aljout half of the respondents in low SES neighborhoods could k 
successfully complete a budget pie involving a mere three budget rfeferas 
(p. 91). Evidence on the difficulty of their type of task also comes 
from a study conducted by Aldrich et. al. ( 1982). Here respondents 
were explicitly asked to choose between inflation and unemployment. A 

large number were either unable or unwilling to make this difficult 

t 

choice. In short, attempts to consjrain respondents by requiring 
explicit trade-^offs may non-randoml^ exclude large numbers of citizens 
from the public in public opinion surveys. 

Finally, the budget pie exercise appear to be a time consuming and 
relatively difficult to administer instrument. This .was even true for 
the very simple version employed by Mclver and Ostrora. A more 
sophisticated version such as the game used by Beardsley e^tj._ajU^ 
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would probably require one or two hours to set up and administer. 
Given the limited range of issue areas that can be covered by this 
technique at any one time, these administrative problems are not 

■ # 

trivial. 

f 

A second non- traditional approach to opinion measurement holding 
some promise for generating more politically relevant data are 
measures that allow respondents to select: -ranges of responses. Ferhaj 
the most developed of Vthese is the one ufeed by Sherif (see,' for \ 
example, Sherif, Sh^if and Nebergall, fn this approach au 

attitude defined in terms of a person's stands .on some o'bjecc, 
issue, person, groups>^ institutions- Stands are further 
categorized in terms of-feone of acceptance, indifference, and 
rejection, y.he inclusion o^ranges in this conception of attitude 
meatus that people who, say, ihare a prefered position on an issue may 
neviertheless disagree on what are unacceptable alternatives* 

The operational implementations of this measurement approach is 
fairly straight fd'rward. For a given issue a range of statement^? 
going frgm one extreme to the other are prepared. On the abort ior| 
issue, for example^ there may be a do^en statements ranging from "No 
restrictions whatsoever*' to "Not permitted under any circumstances." 
No assumptions are made regarding the Intervals between each 
statement, ReponSents are instructed to read through all the 
statements and then sort them into the "accept, "* "reject" and 
"indifferent" categories. 

A somewhat simplier approach to measuring preferences in terms 
of tanges is offered by Aldrich et. al. (1982). Basic^tly, 
respondents were presented with the familiar CPS seven point scale 
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but respondents were allowed to select a Tange of ^f^^^ft^^-they accepted 
(indifference vs, rejection was not considered). Overall, about 
Mlf the respondents made use of a range of point^s at least once 
on questions dealing with relations with Russia and government aid 
to minorities* 

As with the cut a budget pie approach, 'the gafia in relevant 
political data is not cost free. The tet^inique develdped by She 

sir ■• . 

in particular has' several drawbacks, ia term^ of the typical public 

opinion poll. Like the budget pie technique, this technique may bd 

beyond tt^e capacity of poorly educated citizens. We say "may" 

because the ^technique seems to have been used exclusively among 

college populations or members of organisations* It would not. be 

surprising to find that many people have difficulty in iioriing a 

dozen or so statements on a single topic. Both techniques are also 

more time consuming and^diff icult to ^diuinisLer. Recall that a ^ 

respondent must read several sClatements first and then given ample 

time to make cV^anges. , . ' * ^ 

Perh'aps Che b-iggest drawback is the problem of aggregating 

7* these type -of data. Aldrich et. al> , "solve" thi^, probUm^y 

considering only the "accept" portion thp scale, giving position 

numerical values not substantive labels, and using mean scores In ^ 

Statistical calculation (i.e.,- 2-3 becomes 2.5). While pii^rmitting 

quick aggregation, such a solution detracts from the p^^irpose of 

• allowing people to select more than are alternative. ISherif e£. 

X 

al. and others using^this appraoch simply show frequency 
distribution for each issue position. This is perfectly fine for 
showing hjjw the public feels on an issue, but a more conipelx anlaysis 
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is awkward with this wealth of percentage data.^ 

These, then, are some of the Cechrviques that may bring us a few 
steps closer to obtaining politically relevant data on what the 
public wants from government. It should be clear that we remain a long 
way from having satisfactory measures.^ For researchers interested' 
in pfrsuing better, more relevant measures of public preferences, 
two ^reas in particular require future attention. First, as we have 
already indicated, while techniques such as budget pies or range 
type scales maybe a step in the right direction, they are not wiihouT 
their limitations. Considerable tinkering remains to be done. 
Whether these drawbacks can be resolved is very much an open 
question. It is entirely possible that some problems, e.g., 
comprohensibility to poorly educated respondents, are beyond 
realistic solutions. 

Second, efforts must be made to measure certain , aspects of public 
thinking usually ignored by the dominant "attitude- research" 
approach. For example, little work has been done to define the 
limits^of public capacity to deal with certain types of issues. 
Research on attitudes vs. non-attitudes, attitude stability or 
attitude structure does not address the question of whether citizen 
can grasp the complexities and consequences to the policy choices faced 
by officials. This research is crucial if wc are to avoid asking 
citizens for preferences on topics they cannot grasp. V^e also know 
little about what costs (monetary and non-monetary) people are 
willing to pay for policies. It is entirely possible that citizen^ 
preferences depend greatly on perceived costs and "that making people 
aware of these costs caA substantially alter policy demands. 
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Concl usions a nd Specvnat ioj 

<' 

What our analysis has argued is that there is a significant 
gap between the information derived from standard opinion polls and 
the type of data required to make accurate assessments of the 
policy-opinion relationship. A question like: "Should the 
government spend .more (or less) on national defense?" yields* 
little— if any — iWication of what people want. This is true even 
if responses meet all the ^Uandard validity and reliability 
criteria. A politically relevant measure , would show (1) whether 
people grasp what they are spying; (2) will respondents be willing ^ 
to make the trade-offs and priority re-orderlngs that accompany 
their preferences; (3) wHether people will pay the necessary costs; 
and (A) what people are willing to- accept as satsif actor/ outcomes . 
Unless this (and other^ additional information is known, we can 
find ourselves in a situation where public officials are held ^ 
accountable for meeting a public demand that is impossible to meet 
or is meetable in so many .different ways that our assessment of 
consistency is meaningless. 

We are not claiming that existing opinion data tells us 
nothing about popular preferences. Especially where a situation is 
fairly clear-cut, the cohventional poll question can yield some 
relevant information. For example, a long history of surveys 
p^shpwing overwhelming public opposition to mandatory school busing 
to achieve racial integration does convey some real information to 
public officials. But. once we move towards issues such as choices 
among economic policies, changes in government social services 
* policy, tax reform, the present of foreign policy objectives and 
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the like, the information conveyed by traditional polls is of very^. 
limited value* 

The inadequacy of exi^tinfe measures can lead the researcher to 
several lines of inquiry. If one'asslVmes that politically 
relevant measures are^ both possible and desirable, the next step is 
to develop these measures. Rather than continue our methodological 
debates over such problems as question wording, use of DK 
alternatives, acquiescence, response set, question-order effects, 
form of question administration and so on, we could re-focus our 
energies on such problems as how do we get ordinary citizen to 
understand the multitude of costs associated with a given policy 
option. In effect our raethodoiogicai efforts would be directed «u 
determining "what do citizens really want f rora'government" not 
"what do citizens think politically" (the focus, of "attitude 
research"). 

A second line of inquiry begins by assuming that the 
methodology necessary to reveal politically relevant mass preferences 
cannot be developed. This belief may derive from a lack of confidence 
in survey research technology or a very uncharitable view of the quality 
of citizen political thought. If this^is so, how does a researcher 
deal with concepts such as accountability or government 
responsiveness? Are these notions to be abandoned? Ar^» they .to be 
limited to a few issue areas where poll data can be meaningful? Or, m\ 
the other hand, do we substitute election and referendum data for 
opinion poll data? 

A third line of inquiry might be described as analyzing the 
politics of bad data.' Let us assume that pollsters realize that the. 
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usual messages conveyed by -poll results are highly flawed. This 
presentation is not necessarily a conscious f raud-- pollsters honestly 
believe that they are performing a useful public' service with the best 
available techniques. If the data are knovm to be flawed, how|\f,er, the 
question now becomes: Why does this endeavor -continue with little or 
no effort to get more relevant data? What motivates pollsters to ask 
repeatedly questions about increasing or decreasing government programs 
when it is reasonably, certain that many people do not understand the 
program and show little appreciation of the program's costs and 
benefits? Are pollsters simple-minded? Are they guilty of 
exaggerating the wisdom of public thinking? More interesting is the 
possibility that such attention to public opinion, even if- poorly 
direct^^^ serves to reassure people that the public's opinion is being 
given serious attention despite government actions to the contrary. 

' In conclusion, assessing the relationship between whab people want 
.and governmenC policy requires considerably re-thinking. Until- now 
researchers have been conteat to make-do with data poorly suited to . 
tilling us what citizens really wanted. Existing techniques only •>• 
appear^ to yield politically relevant data. Once an analyst begins 
asking questions about what the data really show about what peopl6 
want,, the need for drastic changes in data collection • become obvious. 
Such changes do not involve the perfecting of existing techniques. Our 
contention is that public-opinion researchers have for years operated 
with & inappropriate view of mass political thinking. The goals 
should nat be to measure political attitudes. The goal should be to 
measure political preferences ■ in a way that makes them relevant to . the 
decisions faced^ by public officials. Only then can we make valid 
assertions regarding the responsiveness of government to Its citizens.., 
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